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ANALYSIS  OF  COVARIANCE 
WITH  LINEAR  REGRESSION  ERROR  MODEL 
ON  ANTENNA  CONTROL  UNIT  TRACKING 

By 

DANIEL  T.  LAIRD 


ABSTRACT 

Over  the  past  several  years  DoD  imposed  constraints  on  test  deliverables,  requiring 
objective  measures  of  test  results,  i.e.,  statistically  defensible  test  and  evaluation  (SDT&E) 
methods  and  results.  These  constraints  force  the  tester  to  employ  statistical  hypotheses,  analyses 
and  perhaps  modeling  to  assess  test  results  objectively,  i.e.,  based  on  statistical  metrics, 
probability  of  confidence  and  logical  inference  to  supplement  rather  than  rely  solely  on  expertise, 
which  is  too  subjective.  Experts  often  disagree  on  interpretation.  Numbers,  although 
interpretable,  are  less  variable  than  opinion.  Logic,  statistical  inference  and  belief  are  the  bases 
of  testable,  repeatable  and  refutable  hypothesis  and  analyses.  In  this  paper  we  apply  linear 
regression  modeling  and  analysis  of  variance  (ANOVA)  to  time-space  position  information 
(TSPI)  to  determine  if  a  telemetry  (TM)  antenna  control  unit  (ACU)  under  test  (AUT)  tracks 
statistically,  thus  as  efficiently,  in  C-band  while  receiving  both  C-  and  S-band  signals.  Together, 
regression  and  ANOVA  compose  a  method  known  as  analysis  of  covariance  (ANCOVA).  In 
this,  the  second  of  three  papers,  we  use  data  from  a  range  test,  but  make  no  reference  to  the 
systems  under  test,  nor  to  causes  of  error.  The  intent  is  to  present  examples  of  tools  and 
techniques  useful  for  SDT&E  methodologies  in  testing. 

KEY  TERMS 

Null-  and  alternative-hypotheses,  tracking  mode,  TM,  AGC,  ACU,  Scan  rate,  Slew  rate, 
TSPI  samples,  GPS,  INS,  observables,  calculated,  dataframe,  R,  inner-product,  modeling, 
ANOVA,  ANCOVA,  F-test  and  Mest,  PDF,  CDF. 

INTRODUCTION 

Using  statistical  methods  we  decide  to  either  reject  or  accept  the  null-hypothesis.  Ho  or  its 
alternative  hypothesis,  Hi.  We  do  so  based  on  objective  criteria  derived  from  analyses.  The  test 
null  hypothesis  is: 

Hq:  AUT  tracking  errors  are  statistically  identical  independent  of  data  carrier. 

The  AUT  tracks  on  conically  scanned'*  signal  strength,  or  amplitude  sampled  from 
automatic  gain  control  (AGC)  circuits,  and  on  controllable  conic-scan  (Scan)  and  antenna  slew 
(Slew)  rates.  The  test  includes  S-band  and  C-band  data  channels  (which  carry  TSPI)  of  a  C-band 
tracking  AUT.  Ideally  the  tracking  algorithm  should  have  no  dependence  on  observed  angles  or 
data  band,  i.e.,  the  tracking  algorithm  should  not  depend  on  pointing  angle,  as  the  conic-scan 
center  is  controlled  to  stay  on  peak  tracking  band  signal  strength.  Of  course  this  is  ideal,  and 
we’ve  discovered  that  tracking-error  is  dependent  on  combinations  of  both  azimuth  and 
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elevation,  and  uniquely  so  receiving  in  C-  or  S-band.  We  had  no  information  about  the  ACU 
tracking  algorithm,  only  commanded  and  observed  data,  which  requires  another  source  to 
determine  tracking  error. Data  includes  observed  and  commanded  angles,  AGC  levels  and  the 
rate  control  states.  This  paper’s  focus  is  analysis  of  tracking-error  using  TSPI  data  culled  from 
each  of  the  carriers  at  specific  test  points.  In  a  companion  paper‘d  we  extend  analyses  into 
synthesis  of  an  auto-tracking  model  independent  of  observed  angle.  All  modeling  and  analyses 
is  in  R,  a  statistical  application. 


ANALYSIS  OF  COVARIANCE 

For  modeling  purposes,  received  TM  signal  power  is  a  bounded  continuous  parameter 
sampled  from  receiver  automatic  gain  control  (AGC)  circuits  of  an  integrated  AUT.  Local 
azimuth  and  elevation  angles  are  also  modeled  as  bounded  continuous  parameters,  commanded 
by  and  measured  from  the  AUT  feedback.  An  ACU  employs  two  categorical  tangential^  or  rate 
control  parameters,  designated  as  Scan  and  Slew.  Other  available  data  include  target  TSPI: 
global  position  system  (GPS)  and  inertial  navigation  system  (INS)  data.  Since  carriers  radiate 
from  a  moving  target  that  changes  in  both  global  position  and  attitude,  we  experimented  with 
various  regression  models  with  parameters  from  the  target.  Due  to  length  constraints  we  cannot 
discuss  all  models  and  variations.  Thus,  we  focus  on  a  single  tracking  model  based  on  a  subset 
of  ACU  parameters  and  target  TSPI  (GPS  and  INS)  samples.  Target  TSPI  is  required  for 
calculating  measures  on  tracking  accuracy  of  milli-radian  precision  samples  to  build  a  tracking 
error  model.  ANOVA  reveals  numerical  estimates  of  the  target’s  attitude  on  tracking-error.  We 
construct  a  tracking-error  model  of  a  continuous  expected  error,  linearly  regressed  against  track 
angles,  AGC,  rate  controls  and  a  subset  of  target  attitude.  Aircraft  attitude  yaw,  pitch  and  roll 
were  investigated  as  potential  sources  of  transmitter  radiation  pattern  fluctuation  on  receiver 
AGC  levels.  ANCOVA  employed  a  final  reduced  tracking-error  model.  Paper  length 
limitations  allow  only  brief  discussion  of full  model  reduction. 

LINEAR  REGRESSON  MODEL 

The  auto-tracking  model  is  built  on  a  linear  predictor  of  a  regressed  measure  vector  and 
its  conjugate  observable-control  space,  which  together  form  an  inner-product  space®.  The 
predictor  is  a  functional  form  specifically  an  inner-product: 

P*x  =  Po  +  2  PkXk  =  bx.  (1) 

The  bx  is  a  common  statistical  symbol  for  the  inner-product.  P  designates  predictor  coefficients, 
conjugates  regressed  against  the  sampled  independent  data  variables  x.  Po  designates  the 
predictor  intercept  (the  null-model).  The  response  is  a  continuous  variable  predicted  by  bx.  We 
designate  the  model: 

Imiy  ~  bx).  (2) 

where  y  designates  estimated  error.  Parameters  are  collated  as  a  dataframe  constructed  from 
sample  files.  A  sample  is  shown  in  Figure  1.  We  see  from  the  figure  a  separation  of  dataframes 
per  receiver  band.  The  dataframe  contains  time  common  to  both  ACU  and  TSPI.  Time 
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alignment  correlates  ACU  and  TSPI  samples  at  IHz.  Observed  angles  and  AGC  levels  come 
from  the  ACU,  the  expected  angles  and  line-of-sight  (LoS)  are  derived  via  a  geometric  modef 
that  translates  TSPI  global  position  to  local  azimuth  and  elevation  angles  of  LoS  between  tracker 
and  target.  Rate  control  is  a  state  from  the  ACU.  AT  is  derived  from  the  ACU  elevation  and 
azimuth  tracking  modes.  Attitude  data  is  sampled  from  target  INS.  Two  categorical  tangential® 
control  parameters  are  conical-scan  rate,  Scan  e  {2}'^  and  four-level  slew  rate.  Slew  e 


Figure  1  Samples  Dataffame 


All  categorical  parameters  are  typed  as  factors  for  ANOVA.  Angles  are  bounded  modulo 
360°:  {Az.Obs,  El.Obs,  Yaw,  Pitch,  Roll}  cz  [0,  360]. 


summary  S&C-bands.txt  -  Notepad 
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summary (Sbandl[, 6: 17] ) 

Az.Err  El. Err 


Min. 

-175.611 

Min. 

-87.714 

Min. 

:  0.456 

Min. 

:  0.010 

Min. 

:1.000 

Min.  1  :13.92i 

1st  Qu. 

-23.817 

1st  Qu. 

-5.460 

1st  Qu. ; 

:  6.654 

1st  Qu. 

:  1.840 

1st  Qu. 

:1.000 

1st  QiJ. :  13.92' 

Median 

0.828 

Median 

-2.736 

Median  ; 

:  8.703 

Median 

:  4.430 

Median 

:1.000 

Mediarj  :13.92| 

Mean 

-9.808 

Mean 

-4.605 

Mean 

:  9.943 

Mean 

:  8.125 

Mean 

:1.459 

Mean  '  :13.92j 

3rd  Qu. 

2.817 

3rd  Qu. 

-1.052 

3rd  Qu. ; 

: 14. 188 

3rd  Qu. 

:  7.157 

3rd  Qu. 

:1.000 

3rd  Qii:13.92# 

Max. 

348.351 

Max. 

8.233 

Max. 

: 21. 255 

Max. 

: 45. 330 

Max. 

:5.000 

Max.  \:13.9jf 

Pitch 

Roll 

Scan 

Slew 

AT 

ProbAT  \  / 

Min. 

: -179.84 

Min. 

: -9.156 

Min.  :1.000 

Min. 

1.000 

Min. 

:0.000 

Min.  :0.00^?f56 

1st  Qu. 

, :  -15.11 

1st  Qu . ; 

:  2.484 

1st  Qu. :1.000 

1st  Qu. 

2.000 

1st  Qu. 

:0.000 

1st  Qu. : 0.0138504 

Median 

:  11.85 

Median  : 

:  2.875 

Median  : 1.000 

Median 

3.000 

Median 

:0.000 

Median  : 0.0579423 

Mean 

:  55.81 

Mean 

:  2.789 

Mean  : 1.225 

Mean 

2.837 

Mean 

:0.273 

Mean  : 0.2729529 

3rd  Qu. 

, :  171.69 

3rd  Qu. ; 

:  3.250 

3rd  Qu. :1.000 

3rd  Qu. 

4.000 

3rd  Qu. 

:1.000 

3rd  Qu. : 0.3831964 

Max. 

:  179.59 

Max. 

: 10.438 

Max.  :2.000 

Max. 

4.000 

Max. 

:1.000 

Max.  : 1.0000000 

>  summary(Cbandl[,6:17]) 

Az. 

Err 

El, 

.Err 

LoS 

AGC 

cAGC 

Yaf  \ 

Min. 

: -355.331 

Min. 

: -79.188 

Min.  :  0.622  Min. 

:  1. 

,160  Min 

:1. 

,00  Min.  |:13.92  ' 

1st  Qu. 
Median 
Mean 
3rd  Qu. 
Max. 

Pitch 


-18.751 

0.241 

-5.007 

4.306 

349.406 


Min. 

1st  Qu. 
Median 
Mean 
3rd  Qu. 
Max. 


-179.781 

-20.641 

3.844 

40.032 

169.141 

180.000 


1st  Qu. 
Median 
Mean 
3rd  Qu. 
Max. 

Roll 

Min. 

1st  Qu. 
Median 
Mean 
3rd  Qu. 
Max. 


-4.250 

-1.581 

-2.874 

0.376 

22.449 

-19.562 

2.125 

3.094 

3.071 

4.656 

15.328 


1st  Qu.:  6.601 
Median  : 10. 785 
Mean  : 11. 126 
3rd  Qu. :15.745 
Max.  : 23. 996 
Scan 

Min.  :1.000 
1st  Qu.:1.000 
Median  : 1.000 
Mean  : 1.257 
3rd  Qu. :2.000 
Max.  :2.000 


1st  Qu.:  3.930 
Median  :  7.210 
Mean  :  9.648 
3rd  Qu. :13.360 
Max.  : 51.660 
Slew 

Min.  :1.000 
1st  Qu.:1.000 
Median  : 1.000 
Mean  : 1.689 
3rd  Qu.:3.000 
Max.  : 3.000 


1st  Qu.:1.00 
Median  :1.00 
Mean  :1.48 
3rd  Qu. :2.00 
Max.  :5.00 
AT 

Min.  : 0.0000 
1st  Qu.: 0.0000 
Median  : 0.0000 
Mean  : 0.1549 
3rd  Qu. : 0.0000 
Max.  : 1.0000 


1st  Qu  1:13.92  i 
Median  1:13.92  j 
Mean  |:13.92  | 
3rd  Qu.*:13.92  * 
Max.  113.92# 
ProtJl^T  / 
Min.  : 0^0002933 
1st  Qu.: 0.0280136 
Median  : 0.0534819 
Mean  : 0.1548841 
3rd  Qu. : 0.1958546 
Max.  : 0.9980557 


Figure  2  Dataframe  Summary 
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Note  the  constant  Yaw  in  the  statistical  summary  of  the  data,  figure  2.  This  will 
influence  our  final  model  parameterization.  Automatic  gain  control  is  a  derived  parameter  from 
multiple  receiver  status  samples.  Empirical  decibel  range  samples  are:  {AGC}  cz  0  <  dB  <  52. 
We  generalize  linear  models  for  both  azimuth  and  elevation  errors  in  R  as: 

/w(Az.Err  ~  Az.Obs  +  El. Ohs  +  (c)AGC  +  . . .  +  Pitch  +  Roll)  (3a) 

/w(El.Err  ~  El.Obs  +  Az.Obs  +  (c)AGC  +  . . .  +  Pitch  +  Roll)  (3b) 

Parameters  are  particular  to  each  model,  unique  for  band  and  angle  type  (azimuth  or 
elevation);  thus  we  have  four  models  to  consider.  The  parenthesized  ‘c’  in  (3)  indicates  either 
continuous  (AGC)  or  categorized  gain  control  (cAGC)  in  the  model.  The  details  of  each  model 
are  shown  in  figures  3a  and  3b.  The  categorization  of  AGC  cAGC  adds  no  significance  to  the 
model  predictability,  so  will  not  play  a  part  in  the  final  model. 

The  final  models  are: 

/w(Az.Err  ~  Az.Obs  +  El.Obs  +  AGC  +  Slew  +  Scan  +  Pitch  +  Roll)  (3c) 

/iw(El.Err  ~  El.Obs  +  Az.Obs  +  AGC  +  Slew  +  Scan  +  Pitch  +  Roll)  (3d) 


Figure  3a  Azimuth  Model  Form  and  ANOVA  Results 


We  see  both  antenna  observed  angles  and  target  attitude  contribute  significantly  to  error 
estimation.  This  makes  intuitive  sense  since  the  target  radiation  pattern  is  moving.  Attitude  yaw 
was  eliminated  by  the  model  as  an  insignificant  constant.  This  was  obvious  via  the  statistical 
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summary  of  the  dataframes  (Figure  2  above).  We  assume  yaw  was  not  updating  the  acquisition 
store  for  unknown  causes. 


_|  'C&S-bdnd_LRM.modelsiBiANOVA,t3ft  -  Notepad 


File  Edit  Fortnat  View  Help 


evati on : 

>  summary(El . Err . 1m) 
ca11  : 

ImCformula  =  El.  Err  ~  El.obs  +  Az.obs  +  cAGC  +  Pitch  +  Ro11,  data  =  sbandl) 
Resi dual s : 

Min  IQ  Median  3Q  Max 

-18.497  -1.361  -0.451  1.074  32.496 


coeff 1 ci ents : 


Estimate  std.  Error  t  value  Pr(>|t|) 


(intercept)  -2.430322 
El.obs  -0.716422 
Az.obs  0.0150  56 
CAGC2  3.944317 
CAGC3  4.065816 
CAGC4  8.494049 
CAGC5  15.958711 
Pitch  -0.007295 
Ro11  -0.069751 


0.543233  -4.474  8. 22e-06  *** 

0.007913  -90.536  <  2e-16  *** 

0.001600  9.412  <  2e-16  *** 

0.327669  12.038  <  2e-16 

0.376666  10.794  <  2e-16  *** 

0.408506  20.793  <  2e-16  *** 

0.689453  23.147  <  2e-16  *** 

0.001180  -6.182  8.00e-10  *** 

0.040550  -1.720  0.0856  . 


signif.  codes:  0  0.001  0.01  0.05  0.1  ‘  ’  1 


Residual  standard  error:  3.428  on  1603  degrees  of  freedom 
Multiple  R-squared:  0.8722,  Adjusted  R-squared:  0.8716 
F-statistic:  1368  on  8  and  1603  DF,  p-value:  <  2.2e-16 


>  anova(El . Err . 1m) 

Analysis  of  variance  Table 


Response:  El. Err 

Df  sum  sq  Mean  sq  F  value  Pr(>F) 

El.obs  1  117002  117002  9957.6424  <  2.2e-16  *** 

Az.obs  1  44  44  3.7423  0.05323  . 

CAGC  4  11005  2751  234.1471  <  2.2e-16  *** 

Pitch  1  472  472  40.1954  2.98e-10  *** 

Roll  1  35  35  2.9588  0.08560  . 

Residuals  1603  18835  12 

Signif.  codes:  0  0.001  0.01  0.05  0.1 
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Figure  3b  Elevation  Model  Form  and  ANOVA  Results 


The  regression  coefficients  and  statistics  for  our  tracking-error  model  tracking  in  azimuth 
and  elevation  in  C-band  while  receiving  data  from  S-  or  C-band  carriers  are  shown  in  Figures  3  a 
and  b.  We  see  that  the  ‘best  fit  model’  is  slightly  different  for  the  angles.  Note  that  categorizing 
the  AGC  (cAGC)  for  S-band  samples  yields  a  slightly  better  model,  based  on  the  R-squared 
value  on  less  degrees  of  freedom.  Partitioning  AGC  from  a  single  to  multiple  continuous  levels 
reduces  degrees  of  freedom  in  the  model.  We  often  reduce  models  by  eliminating  or 
recategorizing  parameters  with  a  high  />-value,  or  what  amounts  to  less  information  and 
prediction  power.  These  /7-values  are  not  the  probability  of  error,  but  results  of  a  t-test  on  the 
probability  that  the  modeled  parameter  coefficient  contributes  to  informative  model  results,  e.g., 
is  merely  a  random  variable  with  no  more  explanatory  or  predictive  power  than  the  empty  or 
null-model,  i.e.,  the  model  based  only  on  dispersion  or  deviance  around  the  mean  of  observation 
(the  model  intercept,  Po)-  It  is  convenient  for  explanatory  purposes  that  the  full  C-band  error 
model  is  not  regressed  against  categorized  AGC,  as  doing  so  revealed  no  explanatory  advantage. 
So  we  kept  AGC  as  sole  continuous  parameter. 
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Figure  4a  Azimuth  and  Elevation 
Regression  Coefficients  and  Statistics 


We  also  discovered  via  regression  and  ANOVA  that  Slew  factor  levels  are  statistically 
insignificant  for  elevation  over  both  bands.  Therefore,  azimuth  and  elevation  have  statistically 
different  models,  even  within  a  set  signal  band.  Note  that  each  angle  model  has  very  different 
predictor  coefficients  across  bands,  indicating  the  parameters  influence  in  tracking  are  distinct  to 


6 


band.  The  regression  coefficients  and  statistics  for  C-band  tracking-error  model  are  shown  in 
Figure  4.  Regression  summary  and  ANOVA  tables  reveal  that  S-band  elevation  is  the  most 
difficult  of  the  angle  models  to  fit.  All  t-test  (Pr(>|  t  |)  in  the  summary)  show  statistical 
significance  for  the  respective  parameters,  and  the  ANOVA  results  confirm  this;  but  note  that  the 
ANOVA  F-test  (Pr(>F)  in  summary)  significance  indicate  a  parameter  significance  much  less 
than  the  regression  t-test  significance.  We  also  see  that  Slew  rate  3  has  no  statistical  significance 
in  regression  on  elevation  in  C-band.  These  ANCOVA  results  show  our  reduced  parameters  are 
sufficiently  significant,  enough  so  that  we  can  employ  this  model  to  predict  elevation  error. 
Discussion  with  Air  Force  Test  Center  (AFTC)  Telemetry  Technical  Expert  confirmed  elevation 
as  the  more  difficult  angle  to  track  regardless  of  scanning  and/or  slewing*;  thus  we  deduce  it  the 
more  difficult  to  model,  and  our  results  bear  this  out.  We  also  see  that  both  antennas  observed 
tracker-to-target  angles  and  target  attitude  contribute  significantly  to  error  estimation. 

Az-Err  with  Prediction 
S-band 


Sample 


Ei-Err  with  Prediction 
S-band 


Sample 

Figure  5a  Azimuth  and  Elevation  S-Band  Samples: 
Errors  with  Predictions  (colored) 


'  Rf.  3. 
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The  ‘best  model’  fit  employs  continuous  AGC  for  C-band  samples,  as  this  yields  a 
slightly  better  estimation  of  the  'R-squared'  and  'Residual  standard  error'  values  on  less  degrees 
of  freedom.  Interestingly,  the  value  indicates  that  elevation  is  better  explained  w.r.t.  the  null- 
models  than  is  azimuth  for  both  S-  and  C-band.  All  values  contribute  probable  explanatory  and 
predictive  significance,  and  the  F-statistic  indicates  that  model  explains  well  the  observed,  i.e., 
measured  angle  errors. 


Az-Err  with  Prediction 
C-band 


Sample 


Ei-Err  with  Prediction 
C-band 


Figure  5b  Azimuth  and  Elevation  C-Band  Samples: 

Errors  with  Predictions  (colored) 

Observed  and  predicted  error  from  the  model  are  shown  in  Figures  5a  and  b  for  S-band 
and  C-band  received  data,  respectively.  The  track  profile  comparisons  used  observed  angles  and 
theoretical  angles,  the  latter  derived  from  a  geometric  model.^  The  colored  curve  designates  the 
observed  error  w.r.t.  the  derived  angle;  the  black  is  the  predicted  error  based  on  the  model 


'  Rf.  1. 
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parameters.  It  is  clear  from  Figures  5  that  the  C-band  error  in  azimuth  has  several  large 
differences  from  the  expected  value.  These  errors  were  probably  due  to  the  planar  slewing. 

SUMMARY  AND  CONCLUSION 

This  paper  focuses  on  analyzing  tracking  error  building  an  observation  error  model  and 
ANOVA  to  explore  statistical  sources  of  errors  in  measures,  or  observation.  The  ANOVA  and 
model  results  agree,  showing  not  only  the  tracking  ACU  and  carrier,  but  the  tracked  target 
dynamics  (attitudinal  changes)  contribute  to  measurement  based  on  received  signal  strength  and 
the  state  of  control  of  the  ACU.  Our  analysis  and  models  indicate  that  we  should  reject  the  null 
hypothesis.  Ho,  that  the  AUT  tracks  identically,  within  a  defined  error  tolerance  over  C-band  for 
data  received  in  S-band  or  C-band  carriers  and  accept  the  alternative  hypothesis: 

Hi:  AUT  tracks  statistically  differently  in  C-band  while  receiving  S-band  vs.  C-band  data. 

That  there  is  an  error  in  tracking  is  evidenced  by  the  data,  statistics  and  models  will  be 
developed  further  in  the  companion  paper  that  focuses  on  synthesizing  an  auto-tracking  model 
dependent  on  ACU,  independent  of  both  measured,  or  observed  angles  and  target  attitudinal 
dynamics.  We  also  infer  a  practical  difference,  but  make  no  hypothesis  as  to  cause(s).  We 
investigate  this  further  in  the  third  of  three  companion  papers  -  an  investigation  of  a  logistic 
regression  on  autotrack. 
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